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Abstract 

The  need  for  rapid  access  to  current,  relevant,  and  accurate  information  is  at  an 
all-time  premium  —  especially  for  military  operations.  Many  new  Information 
Technology  (IT)  Tools  were  introduced  to  facilitate  these  information-related 
interactions  in  the  Global  Wargame  2000,  held  at  the  Naval  War  College,  in 
Newport,  RI.  A  web-based  architecture,  called  the  Wargame  Information  Grid 
System  (WIGS),  and  a  collaborative  planning  and  operational  environment,  the 
Information  Work  Space  (IWS),  provided  an  alternative  means  to  communicate, 
collaborate,  and  share  information  among  decisionmakers  than  is  seen  in  today’s 
current  operational  environments.  In  addition  to  WIGS  and  IWS,  the  following 
tools  were  provided  to  participants:  Text  Documents,  the  Knowledge  Wall,  Email, 

Text  Chat,  Voice,  and  Video  Teleconferencing.  Developing  an  understanding  of 
the  implications  and  effects  of  distributed  gaming  was  a  key  goal  of  Global  2000 
with  one  objective  being  to  provide  insight  into  future  operations  for  a  distributed, 
network-centric  joint  force.  A  sub-objective  was  to  obtain  empirical  data  as  to  the 
effectiveness  of  these  new  tools,  and  to  identify  user  defined  enhancements  that 
would  better  meet  the  decisionmakers’  requirements.  In  order  to  collect  the  data 
with  which  to  assess  user  perceptions  of  the  utility  of  IT  Tools,  an  IT  Tool 
Functionality  Questionnaire  was  developed.  This  paper  presents  the  results  of 
analysis  of  data  from  112  respondents’  replies  to  the  questionnaire  designed  to 
assess  the  utility  of  the  IT  Tools  with  respect  to  supporting  the  users  accessing 
and  sharing  of  information,  decision-making,  collaboration,  and  how  they  might 
be  improved  in  future  versions. 

1.  INTRODUCTION 

The  need  for  rapid  access  to  current,  relevant,  and  accurate  information  is  at  an  all-time  premium 
—  especially  for  military  operations.  Moreover,  the  need  for  expeditious  transformation  of  that 
information  into  “actionable”  knowledge  is  increasingly  recognized  by  the  warfighter.  This 
exchange  and  transformation  of  information  to  support  the  military  decisionmaker  was  addressed 
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in  the  Global  2000  Wargame  held  at  the  Naval  War  College  (NWC)  in  Newport,  RI.  Global 
2000  addressed  the  theme  of  Network  Centric  Operations  as  enabled  via  four  “pillars:” 
Information/  Knowledge  Assurance,  Assured  Access,  Effects  Based  Operations,  and  Forward 
Sea  Basing.  Additionally,  a  unique  organizational  architecture  was  played  reflecting  an  overlay 
of  functional  areas  of  responsibility  across  service  components. 

New  Information  Technology  (IT)  Tools,  used  as  part  of  a  networked,  web-based  collaborative 
system  for  command  and  control,  were  also  introduced  in  the  Global  2000  Wargame.  These  tools 
were  provided  to  support  planning  and  operational  processes  by  providing  an  alternative  means 
to  communicate,  collaborate,  and  share  information  among  warfighters  than  seen  in  today’s 
current  operational  environments.  A  key  objective  at  Global  was  to  obtain  empirical  data  as  to 
the  effectiveness  of  these  new  tools,  and  to  identify  user  defined  enhancements  that  would  better 
meet  warfighter  requirements. 

While  information  access  has  always  been  critical  to  success  in  war,  the  concepts  embodied  in 
Network-Centric  Operations  (NCO)  place  an  even  greater  emphasis  on  having  rapid  access  to 
relevant  and  accurate  information.  Concepts  such  as  Information  and  Knowledge  Superiority, 
Knowledge  Management,  and  Effects-Based  Operations,  are  extremely  important  enablers  of 
effective  NCO.  Changing  organizational  relationships  anticipated  for  future  warfare,  expected 
tradeoffs  in  time  and  space,  and  the  need  for  speed  of  effects  with  coordinated  action,  all  point  to 
the  need  for  powerful,  reliable,  and  capable  IT  tools  to  support  the  decision-maker.  Further,  these 
tools  are  expected  to  be  critical  elements  of  success  for  the  warfighter  who  will  be  operating  in  a 
collapsed  battlespace,  working  toward  the  goals  of  achieving  shared  awareness,  information/ 
decision  superiority,  unity  of  effort,  and  the  ability  to  respond  autonomously. 

2.  BACKGROUND 

The  organizational  structure  played  in  Global  was  a  hybrid  of  service  components  crossed  with 
warfare  functional  areas.  Thirteen  functional  areas  (Ground  Control/  Close  Air  Support, 
Operational  Maneuver  From  The  Sea,  Theater  Missile  Defense,  Air  Defense,  Deep  Strike/ 
Interdiction,  Sea  Control,  Special  Operations  Forces,  Rear  Area  Security,  Intelligence,  Sensing, 
and  Reconnaissance,  Fires  Coordination,  Effects,  Information  Warfare,  and  Fogistics)  were 
played.  Responsibilities  for  their  execution  were  passed  to  service  component  commanders: 
NAVFOR  (Navy),  MARFOR  (Marine  Corps),  AFFOR  (Air  Force),  ARFOR  (Army),  and  JSOTF 
(Special  Operations).  These  commanders  functioned  in  dual  roles  as  both  service  component 
commanders  and  warfare  functional  area  commanders  responsible  to  the  Commander,  Joint  Task 
Force  (CJTF).  Participants  were  distributed  locally  at  the  NWC  in  rooms  functioning  as  “player 
cells”  representing  operational  entities,  with  some  250  computers. 

2.1  Distributed  Game  Environment 

A  geographically  distributed  environment  was  provided  enabling  both  Commander  in  Chief 
(CINC)  play  aboard  USS  CORONADO  in  San  Diego,  CA,  and  assessment  of  collaboration 
between  other  remote  locations  using  the  Global  2000  collaborative  tool  suite.  Several  new 
technologies  were  introduced  to  facilitate  interactions  in  this  distributed  environment,  including  a 
web-based  architecture,  called  the  Wargame  Information  Grid  System  (WIGS),  and  a 
collaborative  planning  and  operational  environment,  the  Information  Workspace  (IWS). 
Additionally,  emergent  technologies  such  as  the  Knowledge  Wall  and  CAESAR  II,  were 
introduced  as  collaboration,  decision  support,  and  course  of  action  analysis  tools  The  need  for 


extensive  collaboration  and  coordination  across  functional  areas  and  components  to  accomplish 
the  mission  was  recognized  as  key  to  the  success  of  this  organizational  design. 

2.2  GLOBAL  Wargame  2000  Objectives 

During  Global  Wargame  2000,  several  aspects  of  the  game  were  examined  and  evaluated  by 
various  teams  who  focused  on  knowledge  management,  IT  Tool  utilization,  organizational 
design  and  adaptation,  and  effects  based  operations.  The  implications  and  effects  of  distributed 
gaming  were  also  key  features  of  Global  2000  with  one  objective  being  to  provide  insight  into 
future  operations  for  a  distributed,  network-centric  joint  force.  Additionally,  a  sub-objective  for 
the  game  was  to  explore  the  use  of  tools  to  enable  the  participants  to  self- synchronize  their 
planning,  decisions  and  actions,  and  to  facilitate  clarification  of  the  roles  and  responsibilities  for 
participants  at  each  of  the  distributed  sites.  The  introduction  of  these  new  IT  tools,  a  unique  way 
of  organizing  the  forces,  and  new  processes  to  be  employed  for  conducting  business  made  for  an 
ambitious  agenda  and  provided  a  complex  data  collection  environment. 

2.3  IT  Tool  Use  Assessment 

It  is  important  to  note  that  several  of  these  IT  tools  were  implemented  for  the  first  time  during 
Global  Wargame  2000.  Moreover,  several  of  these  tools  were  included  for  the  express  purpose 
of  testing  their  potential  utility  and  gathering  data  on  how  they  might  be  enhanced.  Therefore,  a 
questionnaire  was  developed  to  help  obtain  information  that  would  be  useful  to  the  developers  of 
the  various  tools  by  collating  feedback  from  a  sizable  sample  of  the  participants.  Most  of  the 
tools  provided  were  in  various  stages  of  development  —  with  some  still  in  the  conceptual  design 
phase.  Thus,  feedback  from  the  game  participants  has  been  taken  as  constructive  feedback  to 
support  developing  improved  tools  to  enable  Network-Centric  Operations. 

2.4  Additional  Analyses 

This  paper  represents  the  results  of  analysis  of  the  data  gathered  at  Global  Wargame  2000  on  IT 
tool  use,  participants’  responses  regarding  tool  limitations,  and  suggested  improvements. 
Additional  analyses  are  required  to  fully  interpret  all  the  data.  One  important  analysis,  would  be 
to  assess  the  participants’  responses  to  the  questions  on  the  IT  tools  while  taking  into  account  the 
various  locations  and  “jobs”  of  the  people  who  rated  the  tools,  and  from  the  perspective  of  the 
intended  audience  of  each  of  the  tools.  It  is  anticipated  that  this  analysis  may  affect  the  ratings  of 
the  tools,  as  are  presented  in  this  analysis.  It  is  recognized  that  participants  who  did  not  have 
access  to  some  tools,  e.g.,  Video  Teleconferencing  (VTC),  yet  rated  them  on  the  questionnaire, 
may  have  rated  their  usefulness  as  “low”  because  they  did  not  use  that  particular  tool.  This 
further  reflects  a  requirement  to  objectively  assess  tool  usefulness  in  terms  of  subjective  utility 
assessments  by  participants  as  to  the  relative  value  of  the  tools’  ability  to  meet  the  different 
priorities  and  mission  objectives  of  players  at  varying  positions  within  the  organization.  With 
respect  to  some  tools,  such  as  VTC,  many  participants  in  the  component  cells  either  did  not  have 
access  or  perceive  a  direct  need  for  its  use. 

3.  IT  TOOL  FUNCTIONALITY  QUESTIONNAIRE 

In  order  to  collect  the  data  with  which  to  assess  user  perceptions  of  the  utility  of  IT  Tools,  an  IT 
Tool  Functionality  Questionnaire  was  developed  by  the  Adaptive  Architectures  for  Command 
and  Control1  (A2C2)  research  team.  The  IT  Tool  Functionality  Questionnaire  was  a  modified 
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version  of  the  questionnaire  that  was  administered  at  GLOBAL  Wargame  1999.  This  question¬ 
naire  was  developed  to  support  an  assessment  of  the  utility  of  the  IT  Tools  with  respect  to 
supporting  the  users’  access  to  and  sharing  of  information,  decision-making,  collaboration,  and 
how  the  tools  might  be  improved  in  future  versions.  The  questionnaire  was  developed  to  obtain 
feedback  on  the  functionality  and  limitations  of  the  tools  provided  for  GLOBAL  Wargame  2000. 
The  questionnaire  consisted  of  six  items  that  asked  the  participant  to  address  the  questions  listed 
below.  This  paper  presents  the  results  of  analysis  of  the  responses  to  these  items. 

1.  Which  tools  were  most  helpful  and  how  were  the  tools  helpful? 

2.  How  did  the  tools  impact  decision-making? 

3.  Rank  the  order  of  the  tools  (on  a  scale  from  1  -  8)  in  terms  of  their  usefulness 
and  provide  comments  on  usefulness  (l=lowest  utility;  8=highest  utility). 

4.  What  were  the  negative  aspects  of  the  tools? 

5.  Rate  the  tools  (on  a  scale  from  1  -  8)  in  terms  of  what  features  you  would  like 
to  see  in  the  design  of  a  collaborative  system? 

6.  How  effective  was  the  pre-game  training? 

3.1  Tools  Included  in  IT  Tool  Questionnaire 

The  following  section  provides  a  brief  description  of  each  of  the  eight  tools  that  were  included  in 
the  questionnaire.2 

3.1.1.  Text  Documents.  Text  Documents  were  a  collection  of  documents  located  within  the 
web-based  gaming  architecture.  These  documents  included  material  such  as  the  Battle  Plan, 
Commander’s  Intent,  Rules  of  Engagement  (ROE),  and  many  others. 

3.1.2.  Knowledge  Wall.  The  Knowledge  Wall  (KW)  was  designed  to  provide  senior  decision¬ 
makers  with  a  common  view  of  game  information  and  to  support  a  novel  model  of  collaboration 
(that  is,  asynchronous  collaboration)  for  the  senior  CJTF  staff.  In  addition,  by  virtue  of  it  being 
installed  for  both  the  CJTF  and  the  Commander,  in  Chief  (CINC),  it  provided  a  means  of 
collaborating  for  the  CJTF  and  CINC  echelons,  as  a  secondary  function. 

The  KW  was  developed  and  included  in  Global  2000  at  the  request  of  COMTHIRDFFT  and 
COMCARGRU  ONE,  who  also  suggested  adaptations  for  use  during  the  Global  2000  Wargame 
to  explore  the  implications  of  Network-Centric  Warfare  for  C4I.  The  KW  comprises  ten  21 -inch 
CRTs,  and  two  large-screen  displays.  The  displays  operate  as  a  single,  integrated  digital  desktop. 
Due  to  its  early  stage  of  development,  the  functionality  of  the  KW  was  limited  in  GFOBAF 
2000.  Conceptually,  the  KW  may  be  viewed  as  a  dynamic  status  board.  The  small,  peripheral 
displays  were  intended  to  provide  summary  information  for  each  of  13  functional  areas  that  were 
identified  through  knowledge  engineering  with  US  Navy  COMTHIRDFFT,  COMCARGRU 
ONE,  and  COMCARGRU  THREE  staff.  (These  functional  areas  correspond  to  the  13  areas  that 
were  played  at  Global.)  Each  summary  display  is  formatted  consistently  with  a  variety  of  tools 
dedicated  to  different  operational  requirements.3 


Aptima),  and  several  universities.  The  A2C2  research  program  is  a  multidisciplinary  effort  to  establish  a  body  of 
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Several  other  tools  were  used  during  GLOBAL  Wargame  2000,  however,  many  of  these  other  tools  were 
developed  for  smaller  sub-sets  of  the  participants  and  thus  were  not  included  in  the  questionnaire. 

Knowledge  Wall  Brochure,  Space  and  Naval  Warfare  Systems  Center,  San  Diego,  (SSD). 


3.1.3  Email.  Email  was  a  standard  commercial-off-the-shelf  email  system. 

3.1.4  WIGS.  The  War-Gaming  Information  Grid  System  (WIGS)  was  designed  to  be  the 
central  source  for  shared  awareness  of  the  game  play.  WIGS  was  developed  to  be  the 
authoritative  data  source  for  game  information.  A  Global  Wargame  2000  interactive  homepage 
was  developed  to  provide  players  with  a  means  of  planning,  communicating,  and  promulgating 
orders.  Promulgation  of  this  information  via  the  web  site  was  intended  to  facilitate  changes  to  the 
Common  Operational  Picture  and  also  provide  for  review  of  players’  decisions,  strategy,  and 
direction  both  during  game  play  and  afterwards.4  The  objective  for  this  web  site  was  to:  (1)  post 
information  for  all  players,  (2)  provide  access  to  analytic  tools  and  to  an  underlying  database  of 
reference  and  briefing  materials,  (3)  facilitate  posting  and  exchange  of  documents,  as  well  as  (4) 
provide  links  to  additional  web  sites,  hosted  within  the  Wargaming  Center,  NWC,  in  order  to 
provide  additional  information  related  to  game  play. 

3.1.5  IWS.  The  Information  Workspace  (IWS)  is  a  collaborative  planning  and  operational 
environment  designed  to  be  used  within  and  among  the  functional  and  service  components 
played  in  Global  2000.  IWS  was  designed  to  facilitate  collaboration  on  those  portions  of  the 
game  where  planning  was  required  to  produce  the  recurring  products  that  were  required  as  well 
as  the  products  produced  in  response  to  the  game  play.5  The  capabilities  afforded  by  IWS  were 
expected  to  be  tailored  by  the  users  to  their  specific  requirements.  Some  functions  provided  by 
IWS  include  real-time  text  chat  and  voice,  both  conducted  over  the  internet. 

3.1.6  Text  chat.  Text  chat  is  a  tool  within  the  Information  Workspace  (IWS),  and 
provides  a  form  of  interactive  communication  where  more  than  one  person  can  join  the  chat 
session  to  collaborate.  All  occupants  within  a  virtual  “room”  (within  the  IWS  tool)  can  view  and 
respond  interactively  to  an  initiator’s  message. 

3.1.7  Voice.  Voice  is  also  a  tool  within  the  IWS  where  a  person  can  talk  with  everyone 
who  is  in  the  same  “room.” 

3.1.8  VTC.  Video  teleconferencing  was  a  standard  commercial-off-the-shelf  system. 

4.  METHOD 

The  IT  Tool  Functionality  Questionnaire  was  distributed  to  participants  in  all  functional  areas 
and  component  cells  at  the  completion  of  the  last  phase  of  Global  Wargame  2000.  Ninety-three 
questionnaires  were  returned  and  the  data  from  these  questionnaires  forms  the  basis  for  the 
results  included  in  this  paper.  Additional  information,  in  terms  of  feedback  provided  by  Naval 
Postgraduate  School  observers  aboard  the  USS  Coronado,  and  information  collected  using  an 
earlier  version  of  the  questionnaire6,  is  also  included  in  the  results  and  recommendations. 

A  brief  analysis  of  a  portion  of  the  data  was  completed  during  GLOBAL  2000  and  was  provided 
to  the  Knowledge  Management  Group.  At  that  time  a  request  was  made  for  further  analysis  of 
the  data  to  look  for  differences  among  the  groups  that  participated  in  the  wargame  and  to 


4  GLOBAL  2000:  The  Distributed  Gaming  Environment.  CDR.  Mike  Waite,  Modeling  and  Simulation. 

Naval  War  College,  Newport,  RI. 

5  From  GLOBAL  2000  Business  Rules,  Naval  War  College,  Newport,  RI. 

6  Questions  in  the  earlier  version  were  formatted  differently.  This  original  version  was  modified  during  Global 
Wargame  2000;  observers  at  San  Diego  did  not  receive  this  modified  version,  thus  the  quantitative  data  was  not 
included  in  this  analysis.  Qualitative  data  from  the  CINC  have  been  included  in  this  paper. 


summarize  the  participants’  comments  that  were  included  as  part  of  their  ratings  of  the  IT  tools. 
The  questionnaires  were  grouped  according  to  components  and  cells  and  formed  the  following 
eight  groups  for  the  analysis: 


GROUP 

Number  in  Group 

CJTF 

34 

AFFOR 

7 

ARFOR 

13 

MARFOR 

8 

NAVFOR 

16 

Reachback 

7 

Others 

8 

(Participants  in  the  “Others”  group  included  CINC,  Anchor  Desk,  NCA,  JOC,  2  JSOTF,  and  2  JTF  J4.) 

5.  RESULTS 

This  section  presents  the  results  of  questionnaire  items  #3,  4,  and  5  for  each  tool.  These  items 
asked  about  the  benefits  associated  with  each  tool,  the  limitations,  and  what  features  were 
desired  in  a  future  collaboration  tool.  Data  from  the  questionnaire  for  all  three  items  will  be 
discussed  under  each  tool.  Responses  to  survey  items  1  and  2  are  not  reported  here  for  the  sake 
of  brevity. 

5.1  Factors  Influencing  Ratings 

Ratings  of  the  tools  were  influenced  by  seven  factors:  (1)  the  position  of  the  participant  within 
the  organization,  in  addition  to  the  tool’s  (2)  functionality  (i.e.,  the  perceived  utility  of  the 
features  offered);  (3)  usability  (i.e.,  how  easy  they  were  to  use);  (4)  reliability  (how  well  the  tool 
functioned  during  Global  2000);  (5)  implementation  (who  the  tool  was  designed  to  support,  that 
is,  certain  tools  were  developed  to  support  a  specific  subset  of  the  participants);  (6)  connectivity 
(who  had  access  to  the  tool  —  this  is  a  critical  variable,  as  many  participants  did  not  always  have 
connectivity  to  all  the  tools);  and  (7)  the  user’s  level  of  proficiency  in  using  the  tool. 

5.2  Differences  in  Information  Needs 

Participants’  operating  in  different  locations  in  the  overall  organization  (i.e.,  different  functional 
areas,  different  warfare  areas,  and  at  different  levels  in  the  hierarchy)  will  have  different  infor¬ 
mation  needs.  These  differences  include  the  specific  types  of  information  required  (e.g.,  weather 
is  critical  to  certain  functional  areas,  and  less  critical  to  others),  the  level  of  detail  desired,  and  so 
on,  and  these  varying  needs  will  impact  the  user’s  ratings  of  the  tools.  For  example,  at  the 
highest  levels  of  the  organization,  the  emphasis  will  usually  be  on  an  integrated  high-level 
picture  of  the  situation,  whereas  at  the  lower  levels  the  decisionmaker  will  often  want  to  focus  in 
on  the  specifics  of  a  local  event  or  the  particular  functional  area  for  which  that  node  in  the 
organization  has  responsibility.  These  varying  emphases,  based  on  differences  in  information 
needs,  appear  to  have  affected  how  different  participants  rated  the  usefulness  of  the  tools,  what 
they  viewed  as  limitations,  and  what  features  they  would  like  to  see  in  a  future  collaboration  tool. 

An  additional  factor  that  affected  ratings  of  the  tools  was  technical  difficulties  encountered 
during  game  play  that  precluded  access  to  the  full  functionality  of  some  tools.  In  thee  cases, 


participants  may,  in  some  cases,  have  had  higher  expectations  regarding  tool  capabilities  and 
utility  than  were  met.  It  appears  that  participants’  ratings  of  the  various  tools’  utility  were 
influenced  by  how  relevant  the  tools  were  to  the  people  making  the  ratings  —  rather  than  simply 
the  overall  utility  of  the  tool.  It  should  also  be  noted  that  user  familiarity  with  tool  functionality, 
capabilities,  and  operating  procedures  varied  dramatically  among  players/users.  Many 
participants  expressed  a  need  for  additional  training  that  would  have  enabled  the  users  to  more 
effectively  employ  the  tools. 

5.4  Overall  Usefulness  of  IT  Tools 

Figure  1  shows  the  results  of  the  respondents’  overall  ratings  when  asked  to  rank  order  the  tools 
on  a  scale  from  least  useful  (1)  to  most  useful  (8)  in  terms  of  the  benefits  to  organizational 
effectiveness  and  their  usefulness  in  accomplishing  the  mission  and  assigned  tasks.  The  mean 
rankings  of  the  tools  fell  into  four  groups.  Email  and  WIGS  received  the  highest  rankings  with 
means  of  6.9,  and  6.5,  respectively,  on  an  eight-point  scale.  These  two  tools  were  ranked  notably 
higher  than  the  rest  of  the  tools.  Text  Documents,  Knowledge  Wall,  and  Text  Chat  received  the 
next  highest  ratings.  These  three  tools  all  were  rated  about  equally  with  mean  rankings  of  4.8, 
4.5,  and  4.6,  respectively,  on  an  eight-point  scale.  Voice  and  IWS  formed  the  third  group,  in 
terms  of  perceived  usefulness,  and  received  rankings  of  3.6,  and  3.4,  respectively.  VTC  was 
ranked  the  lowest  with  a  mean  ranking  of  2.3. 

Rank  ordering  of  tools  in  terms  of  the  benefits  to  organizational  effectiveness  and  their 
usefulness  in  accomplishing  the  mission  and  assigned  tasks.  (l=least  useful;  8=most  useful) 
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Figure  1.  Overall  Mean  Ratings  of  IT  Tools  for  Usefulness. 
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The  following  paragraphs  provide  an  interpretation  of  the  rankings  of  the  tools  based  on  the 
comments  participants  listed  on  the  questionnaires.  All  comments  regarding  the  tools  for  items  3, 
4,  and  5,  which  asked  about  the  benefits,  negative  aspects,  and  features  they  would  like  to  see  in 
a  future  collaborative  tool,  respectively,  are  listed  in  appendices  in  the  technical  report  (Hutchins, 
Poirier,  Hocevar,  Kemple,  Sovereign,  Kleinman,  Entin,  &  Rizzuto,  2000).  As  might  be  expected, 


participants’  comments  were  not  necessarily  consistent.  The  tools  will  be  discussed  in  the  order 
in  which  they  were  ranked,  with  the  highest  rated  tool  discussed  first. 

5.1.2  Email.  All  groups  rated  Email  very  high;  mean  ratings  ranged  from  6.4  to  7.9  on  an  eight- 
point  scale.  Table  2  presents  the  ratings,  by  each  of  the  groups,  for  all  the  tools  assessed  on  this 
questionnaire.  Respondents  indicated  that  familiarity,  reliability,  utility,  and  usability  were  all 
strong  factors  contributing  to  this  tool  receiving  the  highest  ranking  of  all  the  tools. 
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Table  2.  Mean  rankings  of  IT  tools  by  group. 

However,  the  fact  that  Email  was  rated  highest  needs  to  be  interpreted  in  light  of  the  context  in 
which  the  tools  were  used  and  rated,  based  on  anticipated  and  unanticipated  technical  and  objec¬ 
tive  game  dynamics.  Throughout  Global  2000,  Email  was  consistently  and  universally  available 
whereas  IWS  was  not  available  to  all  players,  thus,  Email  may  have  appeared  to  participants  as 
the  most  reliable  form  of  allowing  tasking  between  cell  and  component  commanders.  In  some 
cases,  Email  was  the  only  means  of  communication  for  participants  in  addition  to  direct  personal 
interaction,  (i.e.,  walking  into  another  cell/room  to  directly  communicate  with  another  player). 

5.1.2.1  Benefits  of  Email.  Email  received  the  highest  rating  for  usefulness  because  it:  was 
always  available  and  was  thus  the  most  reliable  way  to  communicate  between  all  cells;  provided 
everything  but  video;  was  fast,  easy  and  familiar;  provided  a  great  paper  trail;  and  was  possible 
to  pass  documents  as  attachments  Another  factor  that  may  account  for  Email  receiving  the 
highest  rating  is  that  users  had  a  high  degree  of  familiarity  with  Email  whereas  many  participants 
had  far  less  familiarity  and  experience  with  the  other  tools.  This  high  level  of  familiarity  with 
Email  minimized  the  “costs”  incur-red  in  using  Email  versus  the  newer,  less  familiar  tools.  These 
costs  may  include  cognitive  workload  in  recalling  tool  functions,  capabilities,  and  operating 
procedures;  additional  communication  and  interaction  requirements  to  align  situation  awareness 
regarding  tool  products,  and  the  additional  time  associated  with  performing  tasks  due  to  lack  of 
proficiency  in  tool  utilization. 

5. 1.2.2  Limitations  of  Email.  The  following  salient  comments  provide  an  indication  of  what  the 
participants  viewed  as  the  drawbacks  associated  with  Email.  Since  Email  was  not  collaborative, 
one  does  not  know  when  it  gets  read;  it  has  limited  distribution  compared  to  web  posting;  when 
Email  was  miss-sent  or  was  time  late  it  was  not  considered;  important  email  can  be  ignored 


because  it  tends  to  be  point-to-point;  some  statements  of  plans  by  components  were  posted  as 
fact  without  higher  approval;  and  it  was  sometimes  difficult  to  decide  what  was  information 
versus  opinion. 

5.1.3  WIGS.  WIGS  received  the  second  highest  ranking,  thus  indicating  that  people  found  it 
to  be  a  very  useful  tool.  Participants  were  fairly  consistent  in  their  ratings  of  WIGS.  The  mean 
ratings,  broken  out  by  the  seven  groups,  ranged  from  5.6  to  6.8  on  an  eight-point  ranking  scale. 

5.1.3.1  Benefits  of  WIGS.  Features  offered  by  WIGS  that  participants  considered  to  be 
useful  are  that  it  provided:  an  effective  way  to  share  information;  rapidly  available  information, 
and  a  tool  that  was  easy  to  use;  access  to  many  information  areas,  including  the  Knowledge 
Wall;  good  navigation  and  good  links;  a  good  “base”  to  go  to,  to  get  situation  assessment  and 
lots  of  information;  the  ability  to  see  what  higher  headquarters  sees  for  information;  and  having 
all  information  on  one  page  was  advantageous. 

5.1.3.2  Usability  Limitations  of  WIGS.  Usability  issues  were  the  main  concerns  expressed 
as  indicated  by  the  following  feedback  concerning  WIGS:  there  were  system  capacity  limita¬ 
tions,  it  was  “mouse  click”  intensive,  it  was  hard  to  know  when  plans  had  been  updated,  users 
needed  to  be  in  two  places  simultaneously,  and  there  were  time  latency  issues.  Additional 
usability  issues  included:  the  lack  of  an  automatic  refresh  for  posted  updates;  it  is  cumbersome  to 
move  between  environments;  it  is  time  intensive  and  hard  to  find  the  really  good  stuff  (e.g.,  Intel 
could  be  hidden  in  many  places);  there  was  no  single  site  to  find  out  how  the  war  was  going;  the 
user  had  to  sign  up  and  boot  to  update  battle  plans;  and  the  user  had  to  search  to  find  changes 
versus  the  system  presenting  information. 

5.1.3.3  Suggested  Improvements  for  WIGS.  Suggested  improvements  for  WIGS  included 
recommendations  related  to  both  usability  and  functionality.  Functionality  issues  included  the 
need  to  present  the  common  operational  picture,  a  means  for  the  user  to  know  when  something 
has  changed,  e.g.,  continuous  situation  summaries;  and  provide  a  single  site  to  track  the  war. 
Usability  issues  suggested  include  providing:  a  better  navigation  system;  a  central,  common  hub 
(this  is  critical);  a  more  compact  system  that  requires  less  “clicks;”  and  a  way  to  made  it  easier  to 
move  between  environments. 

5.1.4  Text  Documents.  Text  documents  received  the  third  highest  ranking  in  terms  of 
usefulness  of  the  eight  tools  included  on  the  questionnaire.  Participants’  rankings  of  Text 
Documents  ranged  from  4.3  to  6.0,  on  an  eight-point  scale  (see  Table  2). 

5.1.4.1  Benefits  of  Text  Documents.  Comments  listed  by  participants  regarding  the 
benefits  of  Text  Documents  include  that  it  provided  baseline  data,  and  archived,  public  access 
information  and  permitted  linking  data  to  graphics. 

5.1.4.2  Limitations  of  Text  Documents.  Usability  and  functionality  issues  were  also  the 
main  concern  regarding  Text  Documents.  Usability  issues  included  that  the  user  had  to  “pull” 
information  versus  the  system  “pushing”  information  to  the  user,  and  it  was  difficult  to  find  and 
track  what  was  needed.  Issues  related  to  functionality  were  that  there  was  too  much  information 
to  read,  and  it  was  difficult  to  keep  track  of  changes  regarding  documents  posted,  e.g.,  ROE  were 
difficult  to  track.  Although  participants  felt  the  documents  provided  good  background  material, 
there  was  too  much  content  to  absorb  and  retain  while  engaging  in  the  scenario.  Other  stated 


limitations  of  Text  Documents  include  that  there  was  no  real-time  update,  it  was  not  always 
current,  and  it  needs  to  include  explicit  Commander’s  Intent. 

5.1.43  Suggested  Improvements  for  Text  Documents.  Improvements  that  participants 
would  like  to  see  for  a  future  Text  Documents  tool  include  usability  and  functionality  issues: 
having  fewer  documents  to  read  during  the  wargame,  keeping  the  documents  updated,  and  then 
making  it  easier  to  track  changes  would  improve  the  functionality,  or  usefulness,  of  this  tool. 
Participants  felt  that  there  was  not  enough  time  during  Global  2000  to  read  all  the  documents  and 
that  some  should  be  provided  to  participants  to  be  read  prior  to  the  exercise  as  a  “read-ahead’ 
package.  Additional  comments  regarding  improvements  for  Text  Documents  include  the  need 
for:  a  Joint  on-line  campaign  plan  developed  before  the  wargame  starts,  an  automatic  save 
capability;  and  a  ROE  timestamp  to  facilitate  tracking  changes. 

5.1.5  Text  Chat.  Text  Chat  was  considered  to  be  useful  for  intra-cell,  analyst-to-analyst  collab¬ 
oration  and  “stove-piped”  activities,  but  was  viewed  as  being  less  useful  for  horizontal/vertical 
(i.e.,  cross-cell)  collaboration.  This  perceived  limited  usefulness  for  cross-cell  collaboration  is 
reflected  in  the  participants’  ratings  of  the  usefulness  of  Text  Chat,  which  ranged  from  3.2  to  5.5, 
on  an  eight-point  scale. 

5.1.5.1  Benefits  of  Text  Chat.  Text  Chat  was  viewed  as  affording  benefits  for  collaborating 
and  planning.  Participants  noted  that  it:  provided  an  excellent  backup  to  email  when  rapid 
response  was  essential;  should  be  the  only  tool  utilized  to  develop  executive  plans;  was  good  for 
real-time  collaborating;  and  it  offered  another  form  of  email  but  better  and  faster 

5.1.5.2  Limitations  of  Text  Chat.  Salient  comments  regarding  what  the  participants  viewed 
as  limitations  to  Text  Chat  pertain  to  the  functionality  of  Text  Chat.  Text  Chat  was:  limited  to 
only  one  area  or  “room,”  thus,  not  it  did  not  support  inter-cell  collaboration  (note  that  this 
contrasts  with  what  some  perceived  as  benefits  offered  by  Text  Chat);  not  archived,  so  previous 
messages  blurred  together;  difficult  to  keep  thoughts  between  the  various  participants  organized; 
cumbersome  with  more  than  a  few  participants;  sufficient  for  stove-piped  activities,  but  not 
supportive  for  coordination  between  participants  in  different  cells;  and  labor  intensive,  such  that 
some  users  required  a  dedicated  operator. 

5.1.5.3  Suggested  Improvements.  Participants  recommended  that  the  following  changes  be 
incorporated  to  Text  Chat:  Include  ALCON  (all  concerned)  in  the  net;  provide  the  capability  to 
keep  a  home  chat  open;  provide  an  automatic  archive  capability  that  can  be  exported  to  limit 
redundant  key  strokes;  make  more  areas  accessible  in  the  same  room;  add  a  “History  join” 
feature  where  the  user  can  see  conversation  before  joining;  and  develop  a  protocol  for  use. 

5.1.6  Knowledge  Wall.  The  Knowledge  Wall  was  viewed  an  excellent  concept  that  needed 
more  maturity.  Participants’  rankings  of  the  Knowledge  Wall  ranged  from  3.5  to  5.9  (see  Table 
2).  The  KW  received  its  highest  ratings  from  participants  who  were  functioning  higher  up  in  the 
organization,  that  is  “Others,”  Reachback,  and  CJTF.  This  may  reflect  that  the  intended 
audience/user  for  the  KW  was  the  CJTF.  Participants  in  the  component  cells  rated  it  lower,  most 
likely  because  they  had  to  provide  information  to  the  KW  (and  the  information  flow  tended  to  be 
one-way,  i.e.,  to  the  senior  echelons). 


5.1. 6.1  Benefits  of  the  Knowledge  Wall.  Selected  comments  made  by  participants  regard¬ 
ing  the  usefulness  of  the  Knowledge  Wall  include  the  following:  if  properly  tasked/managed,  it 
is  an  excellent  tool  to  keep  the  commander  informed;  it  provided  a  good  tool  to  see  a  change  in 
situation  assessment  and  permitted  rapid  information  distribution;  provided  good  information  as 
long  as  it  was  kept  up  to  date;  provided  a  common  source  of  data;  once  we  had  a  clear  idea  of 
content  it  was  great;  and  it  provided  rapid  display  and  access  to  a  lot  of  knowledge. 

5.1. 6.2  Limitations  of  the  Knowledge  Wall.  The  following  comments  provide  an  overview 
of  what  the  participants  viewed  as  limitations  to  the  Knowledge  Wall.  The  main  limitations  were 
that  it  was  labor-  and  time-intensive  to  support,  the  lack  of  interactiveness  and  auto-update,  too 
much  “drilling'’  to  get  situation  awareness,  and  the  one-way  usefulness,  i.e.,  to  the  CJTF. 
Additional  comments  regarding  limitations  include  that  it:  was  a  “data  sponge”  as  it  required 
constant  input;  required  a  person  dedicated  full-time  for  input;  provided  infrequent  Situation 
Awareness  summaries;  needed  a  summary  for  WIGS;  did  not  provide  an  integrated,  fused 
picture;  did  not  have  well-defined  business  rules  such  that  different  cells  used  varying  criteria; 
and  because  all  pages  could  not  be  displayed  at  once,  no  unified  picture  was  presented. 

5.1. 6.3  Suggested  Improvements  for  the  Knowledge  Wall.  Improvements  listed  as 
desired  for  the  Knowledge  Wall  include:  the  need  to  update  the  information  more  frequently,  it 
was  too  rigid — need  to  know  where  to  find  the  information  vs.  information  being  there,  it  was  a 
menu  more  than  providing  specific  information.  Additional  comments  regarding  improvements 
include  the  need  for:  updating  the  information  more  frequently;  better  manipulation  tools; 
providing  a  standard  for  red,  yellow,  green  to  facilitate  interpretation;  displaying  an  integrated, 
fused  picture;  smart  “pull”  from  an  integrated  workstation;  design  around  Joint  Doctrine 
approved  functional  areas;  and  providing  updated  alerts,  with  prioritization. 

5.1.7  Voice.  Many  participants  indicated  they  did  not  use  Voice  due  to  problems  with  the 
system’s  availability  and/or  reliability.  As  with  comments  received  on  the  other  tools,  there  was 
some  contradictory  feedback  across  players,  such  as  “Essential  for  direction”  and  “Could  be  very 
helpful,”  which  contrasts  with  “Email  and  Text  Chat  served  the  purpose  well.”  Participants’ 
rankings  of  Voice  ranged  from  3.1  to  5.4,  indicating  they  felt  it  was  moderately  useful. 

5.1.7.1  Benefits  of  Voice.  Of  those  who  did  use  this  tool  within  IWS,  the  benefits  perceived 
to  be  provided  by  Voice  included:  it  was  the  fastest  way  to  pass  time-critical  information  and 
was  a  good  to  clear  up  questions  when  an  issue  was  not  clear. 

5.1.7.2  Limitations  of  Voice.  The  following  comments  concerning  usability  issues  provide 
an  overview  of  what  the  participants  viewed  as  limitations  to  Voice.  There  was  no  record  of  a 
conversation,  thus  it  can  be  misinterpreted;  it  was  difficult  finding  the  person  you  desire;  it  was 
too  hard  to  understand  /  hear  and  this  created  confusion;  ALCON  were  not  in  net;  and  it  was  tied 
to  a  headset. 

5.1.7.3  Suggested  Improvements  for  Voice.  The  suggested  improvements  regarding  Voice 
related  to  enhancing  its  usability  and  include:  it  needs  on-screen  “flag”  to  alert  for  new  voice 
traffic;  it  should  provide  a  multi-point  capability;  and  make  it  so  voice  is  continuous. 

5.1.8  IWS.  Due  to  limited  availability  during  game,  IWS  was  not  used  by  many  participants 
play  although  some  indicated  they  felt  IWS  offered  potential.  Comments  regarding  the  IWS 


include:  collaborative  planning  still  requires  a  higher  authority  for  final  decisions  in  the  MPP; 
participants  should  have  been  forced  to  use  it;  and  participants  felt  that  privacy  leads  to  “stove¬ 
piping”  of  information.  Participants’  rankings  of  Voice  ranged  from  2.1  to  4.1  (see  Table  2). 

5.1. 8.1  Benefits  of  IWS.  IWS  was  viewed  as  useful  for  collaboration,  as  noted  by 
comments  such  as:  it  provided  a  useful  environment  for  collaboration  and  we  need  this  for 
collaborative  planning. 

5.1. 8.2  Limitations  of  IWS.  The  following  comments  provide  an  overview  of  what  the 
participants  viewed  as  limitations  to  IWS.  Functionality  issues  were  that  IWS  was  not  operation¬ 
ally  focused,  it  was  too  “stove-piped”  in  that  it  provided  limited  simultaneous  capability  to  be  in 
more  than  one  “room”,  and  it  offered  no  record  capability.  A  reliability  issue  was  that  it  was  too 
sensitive  to  network  loading  and  thus  there  was  an  inability  to  handle  all  the  people.  Usability 
issues  include  that  IWS  limited  the  user  to  one  room  or  location  at  a  time;  it  was  frustrating  to 
use  without  a  battle  rhythm  or  decision  point  laid  out;  it  was  much  more  complicated  to  share 
data  than  in  email;  and  to  review  a  new  system,  operators  were  required  to  return  to  home. 

5.1. 8.3  Suggested  Improvements  for  IWS.  Recommendations  regarding  improvements  for 
IWS  include  that  it  needs:  to  be  more  flexible,  in  terms  of  allowing  the  user  to  particularly  “hop” 
from  room  to  room;  to  divide  the  server  and  provide  a  means  to  coordinate  planning;  strict 
business  rules  for  user  interaction;  a  nested  collaboration  capability  to  keep  systems  up  longer; 
better  graphic  management;  more  areas  accessible  in  the  same  room;  the  ability  to  locate  a 
person  w/o  paging  them;  and  incorporation  of  maps  into  IWS. 

5.1.9  VTC.  The  VTC  tool  received  the  lowest  rating  on  usefulness  based  on  participant 
responses.  While  the  ability  to  have  eye  contact,  observe  the  speaker’s  body  language,  and  hear 
the  inflection  in  the  speaker’s  voice  appeared  to  be  intuitively  valued  for  building  trust  in  the 
received  message  by  some  respondents,  other  factors  appear  to  affect  how  the  tool  was  rated. 
These  other  factors  appear  to  include  the  participant’s  position  in  the  organization,  availability/ 
access  to  the  tool,  and  prioritization  of  the  value  of  this  tool  relative  to  other  resources.  It  should 
be  noted  that  technical  difficulties  including  audio  problems  prevented  participants  from 
following  the  flow  of  information  during  some  VTCs  and  this  likely  contributed  to  VTC 
receiving  low  ratings.  Participants’  rankings  of  VTC  ranged  from  1.7  to  2.4  (see  Table  2), 
indicating  a  perception  of  low  usefulness  across  respondents.  However,  preliminary  analysis 
indicates  that  perceptions  of  utility  for  VTC  may  vary  significantly  based  on  user  position  in  the 
organization. 

5.1. 9.1  Benefits  of  VTC.  The  only  benefit  listed  by  the  participants  was  that  eye  contact 
and  personal  cues  available  with  VTC  helps  build  trust. 

5.1. 9.2  Limitations  of  VTC.  Participants,  across  the  board,  indicated  the  technological 
difficulties  encountered  during  the  wargame  with  VTC  detracted  seriously  from  its  use  and 
appreciation  of  the  potential  features  it  offered.  The  following  comments  provide  an  overview  of 
what  the  participants  viewed  as  limitations  to  VTC:  it  delays  progress;  it  is  receive  only;  and 
while  it  was  good  for  (high-level)  decision  makers,  it  added  little  for  the  operators. 


6.  View  from  USS  CORONADO 


The  KW  was  the  one  tool  that  was  designed  specifically  for  the  CINC  who  was  located  onboard 
the  Third  Fleet  Flagship,  USS  CORONADO.  In  addition  to  feedback  on  the  KW  received  from 
the  participants  at  the  Naval  War  College,  participants  on  board  USS  CORONADO  provided 
assessments  and  feedback  regarding  their  reactions  to  the  tools  provided  during  the  game.  Some 
specific  comments  from  the  CINC  regarding  the  KW  display  include  those  listed  below.  Most  of 
these  comments  refer  to  additional  information  the  CINC  would  like  to  have  displayed  on  the 
KW  in  future  iterations  of  the  KW.  Other  comments  reflect  usability  issues. 

•  What  is  the  attrition  of  RED? 

•  What  is  the  DEFCON  for  what  area? 

•  Which  strait  is  the  Nimitz  coming  through?  (Add  PIM  (projected  intended  movement) 

of  carriers  on  KW) 

•  What  other  carriers  and  ATF  (amphibious  task  force  units)  are  due  in  the  area?  (Add 

schedule  to  KW) 

•  Why  is  the  EFFECTS  display  RED  in  the  summary  but  not  in  the  detailed  display? 

•  Need  a  better  way  to  correlate  the  colors 

•  In  scanning  the  KW,  there  isn’t  much  that  can  be  done  about  today;  thus,  the  emphasis 

should  be  on  tomorrow  and  the  long-term  picture 

•  Why  are  some  RED  today  and  YELLOW  tomorrow? 

•  Prior  to  C-day  did  the  CJTF  employ  Flexible  Deterrent  Options  (FDOs)? 

•  Needs  real-time  assessment  of  Effects:  unintended  consequences,  in  particular, 

need  quick  fixes 

•  TAPS  model  is  fantastic,  but  not  timely 

7.  Summary  of  Analysis  of  Feedback  on  IT  Tools 

The  responses  to  the  IT  Tools  Questionnaire  indicate  the  participants  generally  felt  that  all  the 
tools  offer  potential  and  all  have  room  for  improvement.  The  majority  of  the  comments  indicate 
that  users  want  tools  that  are  more  reliable,  powerful,  and  developed  to  be  more  user-friendly 
and  intuitive  to  use.  In  general,  there  was  a  learning  curve  involved  before  some  tools  could  be 
fully  utilized  by  the  participants;  as  people  develop  greater  familiarity  and  capability  in  using  the 
tools,  their  impressions  may  become  more  favorable. 

Some  general  types  of  feedback  that  were  made  in  response  to  several  of  the  tools  and  about  the 
wargame  in  general  are  summarized  below.  Reliable,  interactive  systems  with  the  range  of 
attributes  that  were  intended  would  represent  a  good  first  iteration  of  a  collaborative  system  to 
support  joint  command  and  control  decisionmaking.  These  tools  need  to  be  integrated  so  the 
decisionmaker  does  loose  the  picture  by  moving  in  and  out  of  each  area 

7.1  Additional  Capabilities  Desired  in  Tools 

Participants  expressed  the  desire  to  have  a  better  way  to  know  Commander’s  Intent  in  terms  of 
making  Commander’s  Intent  more  understandable.  A  graphical  representation  of  this  informa¬ 
tion,  depicting  the  overall  battle  picture  embedded  within  the  context  of  information  critical  to 
commander’s  Intent,  would  assist  in  users  developing  and  maintaining  good  situation  awareness. 
In  addition,  more  powerful  graphics,  robust  enough  to  present  large  volumes  of  data  on  one 


display,  is  another  capability  the  participants  listed  as  important.  Graphical  presentations  are 
viewed  as  a  feature  that  would  enhance  the  decisionmaker’s  ability  to  understand  the  situation. 
For  example,  including  a  graphic  depiction  of  information  during  briefings,  e.g.,  Order  of  Battle 
tables  were  stated  to  be  too  difficult  to  follow.  More  maps  were  also  listed  as  an  additional 
capability  that  would  be  useful,  in  particular  maps  with  threat  locations  and  labels  of  prominent 
features  and  regions. 

8.  Tools  Desired  in  a  Future  Collaboration  System 

Participants  rank  ordered  the  eight  tools  included  in  the  questionnaire,  in  terms  of  which  tools 
they  would  like  to  have  in  a  future  collaboration  tool.  The  overall  rankings  for  the  tools  are 
shown  in  Figure  2.  Participants  varied  considerably  in  their  response  to  item  #  5,  that  asked  them 
to  rate  the  tools  —  and  the  features  —  they  would  like  to  have  in  a  future  collaboration  tool. 

Rank  ordering  of  the  tools  in  terms  of  what  features  would  you  like  to  see  in  the  design  of  a 
collaborative  system?  (1  =  least  useful;  8  -  most  useful) 


Doc  Wall  Chat 

Figure  2.  Mean  ratings  of  tools  desired  in  future  collaboration  system. 

Table  3  breaks  out  participant  ratings  of  the  tools,  in  terms  of  the  features  they  would  like  to  see 
in  the  design  of  a  future  collaborative  system,  by  the  seven  groups.  WIGS  received  the  highest 
ranking  by  three  groups,  was  rated  second  highest  by  three  groups,  and  third  highest  by  one 
group.  WIGS  was  rated  as  one  of  the  top  three  tools  by  all  seven  groups.  The  general  consensus 
was  that  WIGS  provided  a  critical  information  source.  However,  remaining  questions  exist  as  to 
whether  WIGS  can  be  designed  to  be  more  user-friendly  and  whether  its  functionality  can  be 
improved  (such  as  making  it  less  cumbersome  to  use  and  less  time  intensive  to  use).  WIGS  was 
viewed  by  respondents  as  a  very  effective  tool  to  facilitate  easy,  fast  accessing  and  sharing  of 
many  information  areas  and  as  a  vehicle  to  develop  situation  assessment. 


Email  was  rated  highest  by  three  groups,  was  rated  second  highest  by  one  group  and  third 
highest  by  one  group.  The  features  provided  by  email  made  it  rank  in  the  top  three  rankings  by 
five  of  the  seven  groups.  IWS  was  the  highest  ranked  tool  by  one  group  and  the  second  highest 
ranked  tool  by  two  groups.  Text  Chat  was  the  next  tool  with  the  highest  rankings,  which  was 
ranked  second  highest  by  one  group,  third  highest  by  two  groups  and  fourth  highest  by  one 
group.  Text  Documents  were  ranked  as  the  third  most  important  tool  by  two  groups.  The 
Knowledge  Wall  was  ranked  the  fourth  highest  tool  by  five  of  the  seven  groups. 

Table  3.  Rank  Order  of  Desired  Tools  for  Future  Collaboration  Tool  by  Group* 


CJTF 

AFFOR 

ARFOR 

MAR- 

FOR 

NAV- 

FOR 

Reach 

back 

“ 

Others 

Text  Docs 

4.0 

4.0 

4.7 

6.0 

3 

3.7 

6.0 

3 

4.2 
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4.8 

4 

4.0 

4.8 

4 

4.7 

4 

4.5 

5.8 

4 

5.3 

4 

Text  Chat 

4.5 

6.7 

2 

4.9 

4.7 

4 

5.2 

3 

4.3 

5.4 

3 

Email 

5.6 

3 

7.2 

1 

6.4 

6.4 

2 

4.6 

6.7 

1 

3.7 
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4.6 

5.7 

4 

5.4 

3 

4.7 

4 

4.8 

4 

5.5 

5.2 

WIGS 

6.5 

l 

6.5 

3 

5.5 

2 

6.5 

1 

6.1 

2 

6.4 

2 

6.4 

1 

IWS 

6.0 

2 

4.7 

4.7 

4.6 

6.5 

l 

5.7 

6.2 

2 

VTC 

2.5 

2.0 

4.2 

2.8 

1.7 

3.7 

2.2 

*Note:  Numbers  in  the  lower  right-half  corner  of  the  cells  indicate  the  ranking  for  each  of  the  four 
highest-rated  tools  by  all  the  groups. 


9.  Training 

Based  on  participant  feedback,  additional,  comprehensive  training  is  needed  on  tools, 
individually,  and  then  within  the  context  of  the  organization  and  a  scenario.  Participants  felt  a 
need  to  develop  a  degree  of  proficiency  in  using  the  tools  prior  to  actual  game  run  so  that  there 
are  not  too  many  new  things  vying  concurrently  for  their  attention.  Participants  could  be  given 
time  to  engage  in  a  “mini- scenario”  where  they  can  practice  all  of  the  behaviors  they  will  need  to 
perform  in  the  actual  game,  but  without  the  pressure  of  still  trying  to  develop  proficiency  while 
responding  to  the  mission  requirements  of  the  Global  wargame  scenario. 

9.1  Additional  Training  Would  be  Beneficial 

In  order  to  develop  a  certain  level  of  proficiency,  in  general,  participants  felt  a  need  for  enough 
time  to  perform  (practice)  the  various  actions  in  an  interactive  manner  prior  to  engaging  in  the 
scenario,  as  opposed  to  just  being  shown  how  to  do  certain  things.  A  tiered  type  of  training  may 
help  participants  develop  greater  proficiency.  Tiered  training  would  consist  of:  (1)  an  intro¬ 
ductory  lecture  on  the  tools,  the  function  for  which  each  tool  was  developed  and  the  associated 
business  rules;  (2)  hands-on  practice  in  using  the  tools  individually;  (3)  in  situ  practice  in  a  group 
setting  where  participants  practice  what  they  learned  about  the  tools  in  an  interactive  setting. 


9.2  Training  Recommendations  from  Participants 


Comments  from  participants  were  quite  consistent  in  regard  to  the  training  provided  for  Global 
2000.  They  recommend  that  the  pre-game  training  should  be  mandatory  for  all  players  and  that 
more  time  should  be  dedicated  to  training.  A  greater  emphasis  on  “hands-on’  practice  for  all 
tools  was  stated  as  being  needed  in  addition  to  the  lecture/briefing  style  of  training. 

10.  Conclusions 

The  tools  provided  during  Global  2000  had  a  major  impact  on  the  ability  of  the  participants  to 
access  and  share  information,  develop  and  maintain  situation  awareness,  collaborate  across 
functional  areas  and  component  cells,  and  coordinate  both  vertically  and  horizontally  within  the 
organization.  A  good  deal  was  learned  about  the  tools  and  ways  to  improve  the  tools  for 
developing  a  future  collaborative  system.  The  following  section  summarizes  the  main  lessons 
learned  regarding  the  IT  Tools  used  at  Global  2000. 

10.1  Lack  of  a  Synthesized  Picture 

A  major  theme  expressed  by  the  participants  throughout  the  organization  was  the  need  for 
synthesized  presentations  of  information,  both  within  the  various  tools  used  and  across  the  tools. 
Instances  where  “information”  from  different  tools  was  contradictory  illustrate  that  there  is  a 
need  to  “manage”  the  information  generated  by  the  tools.  (For  example,  in  one  situation  when 
JFLEX  (Joint  Force  Level  Execution  Program)  was  brought  up  the  situation  indicators  showed 
all  GREEN  at  the  same  time  the  KW  had  lots  of  situation  buttons  coded  RED.)  Many  comments 
indicate  that  the  participants  really  want  a  more  synthesized  picture  of  the  current  situation. 

10.2  Lack  of  Bandwidth 

A  second  major  theme  was  the  lack  of  sufficient  bandwidth  for  all  the  tools  to  function  as 
planned.  This  was  a  major  impediment  to  accomplishing  the  game  with  the  given  organization 
structure  and  the  “envisioned”  roles  and  functions  for  the  various  participants. 

10.3  Provide  an  Updated  Picture  of  the  Battle 

An  overarching  theme  across  the  comments  made  regarding  all  the  tools  was  the  need  for  a 
common  picture  of  the  situation.  This  suggested  improvement  applies  both  within  some  of  the 
tools  and  across  the  tools. 

10.4  Need  Business  Rules 

Business  rules  were  requested  to  provide  a  standard  protocol  for  using  the  tools  and  for 
interpreting  the  information  presented.  General  business  rules  are  needed  to  provide  guidance  on 
conducting  operations  in  the  new  operational  milieu  envisioned  in  a  network-centric 
environment.  System  specific  business  rules  are  needed  regarding  when  and  where  to  post 
information,  how  to  update  and  authenticate  information,  and  to  increase  efficiency  by  reducing 
the  need  for  decision-makers  to  search  through  tools  and  data  to  extract  the  needed  information. 
Making  the  tools  easier  to  use  will  also  reduce  the  cognitive  workload  for  the  user,  thus  freeing 
up  finite  cognitive  resources  for  the  decisions  that  require  the  human  decision  maker  and  thus 
increasing  “speed  of  command.” 

10.5  Develop  Tools  that  are  More  Intuitive  to  Use 


Many  systems  appear  to  require  an  interface  that  is  more  intuitive  for  the  user,  who  is  often 
required  to  make  critical  decisions  under  time-compressed  and  ambiguous  conditions.  These 
stressful  decision-making  environments  require  tools  that  are  truly  supportive  of  the  user,  so  the 
user  doesn’t  have  to  “fight”  the  tools,  in  addition  to  fighting  the  actual  battle.  For  example,  an 
intuitive  browsing  capability  would  enable  users  to  access  more  information,  more  rapidly.  A 
tool  that  is  designed  to  be  intuitive  to  use  will  reduce  the  training  time  required  and  will  increase 
the  amount  of  support  it  supplies  to  the  decision-maker. 

10.6  Need  Additional  Training 

There  were  observations  throughout  the  cells  of  instances  where  players  did  not  know  how  to  use 
the  tools  to  sufficiently  to  realize  the  inherent  capabilities  of  the  tools.  There  were  also  instances 
where  players  did  not  know  where  to  go  to  get  information  from  the  “system.” 

11.  DISCUSSION 

11.1  IT  Tools  and  Network  Centric  Warfare 

IT  Tools  are  only  one  element  of  a  complicated  set  of  new  processes,  ways  of  organizing,  and  a 
netted  infrastructure  that  will  fundamentally  alter  how  warfare  and  military  operations  are 
conducted  in  the  future.  As  the  concepts  embodied  in  NCO  continue  to  develop,  these  concepts 
will  be  enabled  by  a  variety  of  decision  support,  analysis,  troubleshooting,  and  adaptation  tools. 
It  will  be  important  to  develop  these  concepts  —  and  the  tools  that  will  support  the  concepts  — 
in  a  coordinated  way  that  will  achieve  the  synergy  that  is  envisioned.  The  criticality  of 
integrating  all  these  new  elements  to  ensure  a  good  match  between  the  processes  to  be  employed, 
the  new  ways  people  will  be  organized,  and  technology  is  exacerbated.  The  transformation  of 
information  to  knowledge  further  increases  reliance  on  the  human  element  as  the  critical  element 
in  the  system  of  systems  in  terms  of  the  way  decision  superiority  can  result  from  the  cognitive 
processes  enabled  by  technology.  A  fundamental  element  of  success  in  future  military  operations 
may  be  the  realization  that  the  “network”  in  network  centric  warfare  is  the  network  of  people 
engaged  in  mutually  supporting  activities  to  achieve  a  common  set  of  goals  using  an  integrated 
collaborative  set  of  tools. 

11.2  Principles  of  Collaborative  Command  and  Control 

Command  and  control  principles  are  fairly  well  established  (such  as  centralized  planning  and 
decentralized  execution,  feedback  loops,  unity  of  command,  unity  of  effort,  etc.),  yet  the 
evolution  of  information  sharing  and  collaborative  planning  technologies  have  potentially  deep 
implications  for  command  and  control.  Network-Centric  Operations  may  introduce  entirely  new 
principles  or  require  the  modification  of  old  principles.  Without  articulation  of  new,  clear 
principles,  it  is  difficult  to  proceed  with  the  design  of  processes,  codification  of  business  rules 
and  the  other  steps  necessary  to  provide  a  modern  command  and  control  system.  Moreover, 
military  requirements  are  arguably  different  from  commercial  information  requirements  and  the 
systems  that  are  produced  to  support  those  requirements.  Differences  include  dynamic 
reconfiguration,  survivability/  redundancy,  and  adaptability,  at  the  least.  Collaboration  must  be 
conducted  within  these  requirements  and  the  essentials  of  the  military  environment. 


12.  RECOMMENDATIONS 


Based  on  the  analysis  presented  in  this  paper,  we  suggest  a  dialogue  on  the  nature  of  the 
principles  of  Collaborative  Command  and  Control.  These  principles  would  serve  to  support 
decision  superiority  in  a  network-centric  environment.  A  “strawman'’  set  of  draft  principles 
should  provide  the  foundation  for  the  first  step  —  which  is  to  stimulate  discussion.  The 
strawman  set  of  principles  would  also  be  useful  for  identifying  issues  for  experimentation, 
formulation  of  new  concepts,  and  general  guidance  for  system  designers. 

An  alternative  approach  would  be  to  identify  groups  of  existing  principles  of  military  C2  and 
prepare  discussion  papers  on  whether  they  may  be  impacted  by  the  advances  in  information 
technology.  Participants  would  indicate  the  degree  and  direction  of  modification.  Then,  an  initial 
document  could  be  prepared  on  the  modified  principles.  A  second  alternative  approach  would  be 
to  start  from  a  fresh  viewpoint  regarding  the  requirements  emerging  from  the  need  to  collaborate 
that  have  been  identified,  largely  from  commercial  practice.  Then  the  additional,  specific 
requirements  levied  by  the  needs  of  a  military  environment  and  the  possible  adjustments  that 
would  be  needed  to  support  information  sharing  and  collaboration  would  be  specified.  This 
information  regarding  specific  military  needs  would  be  integrated  to  provide  a  draft  of  principles 
for  the  design  and  use  of  information  technologies  to  support  Collaborative  Command  and 
control. 
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